docs(skills): improve drafting skills from signal log patterns 2026-08-03 - #454
docs(skills): improve drafting skills from signal log patterns 2026-08-03#454oz-by-warp[bot] wants to merge 6 commits into
Conversation
Co-Authored-By: Oz <oz-agent@warp.dev>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
I'm starting a first review of this pull request. You can view the conversation on Warp. I completed the review and no human review was requested for this pull request. Comment Powered by Oz |
There was a problem hiding this comment.
Overview
This PR updates the docs drafting skill and feature/guide templates to require source or live-product verification for user-facing labels and flags, prefer durable behavior over ephemeral UI details, state availability limits, cover team/admin effects for integrations, and strengthen related-page expectations.
Concerns
- No blocking concerns found in the attached diff. The supplemental security pass found no changed code or configuration introducing security-relevant risk, and
spec_context.mdreports that no approved or repository spec context exists for spec-drift review.
Verdict
Found: 0 critical, 0 important, 0 suggestions
Approve
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
…r skill Co-Authored-By: Oz <oz-agent@warp.dev>
…4-monitor skill" This reverts commit d837a53.
| - **Document durable behavior, not ephemeral chrome** — Prefer workflows, shortcuts, and outcomes that stay true when styling shifts. Drop glyph colors, pixel-level layout narration, and other pure presentation detail unless the reader must recognize them to succeed. ✅ "Press `Ctrl+C` once to stop the in-progress response." ❌ a full inventory of pending/running/failed glyph colors | ||
| - **State availability honestly** — If a capability is preview-only, platform-limited, interactive-only, or not yet in cloud agents, say so next to the claim. Never describe limited-preview behavior as generally available. ✅ "Linux post-processing adds smart cut; macOS applies a uniform speedup." ❌ listing smart cut as a property of every recording | ||
| - **Cover team-wide and admin effects** — For integrations and team features, state who can install, whether every teammate gets access immediately, and any per-user auth or admin steps on the external system. ✅ "A Jira admin must install the app; each teammate links their own account for run attribution." ❌ setup steps that only describe the installer's happy path | ||
| - **Related pages are required on new pages** — End new feature, integration, and guide pages with a Related pages (or Next steps) section. Link siblings and the natural next workflow so the page does not dead-end. ✅ Integrations overview + Agent Management Panel + a sibling Slack/Linear page ❌ no cross-links after Troubleshooting |
There was a problem hiding this comment.
This rule may be too strict for thin and newer stub pages that will be expanded on later. Update rule to include some flexibility around this. If there a logical and valuable next step exists, we should include that on the docs page.
Resolves the open question on source verification: keep the escape hatch for drafts written without warp-internal/warp-server access, but make it non-skippable. Unverified UI labels, flags, defaults, and eligibility claims must be marked inline and surfaced in an Unverified claims section of the PR description (or in the response for local sessions). Co-Authored-By: Warp Agent <agent@warp.dev>
|
Superseded by #487, which consolidates this PR together with the other stacked All four edited Root cause: the agent's schedule used |
…ements (#487) * docs(skills): reduce automation noise and consolidate drafting improvements Recurring docs agents were producing more PRs and Slack messages than the team could absorb. Three systemic causes, plus a batch of GitBook-era migration artifacts that left several skills unable to run as written. Shared conventions (skill-authoring-guidelines.md): - Add "One standing PR per automation": stable branch and title, look before creating, add to the existing PR rather than opening another. - Invert "Slack notifications" to actionable-only. The old rule required posting on every run and was the direct cause of the channel noise. Its silent-failure rationale is preserved by requiring a run log instead. - Rewrite "Log availability" so outer loops read the log branch rather than main, and never merge the standing log PR as a workflow step. Cron correctness: - `0 17 1-7 * 1` is not "first Monday". Cron ORs day-of-month with day-of-week, so it fired ~11 times a month and produced four conflicting PRs in six days. Replace with `0 17 * * 1` plus an in-skill first-week guard in improve-drafting-skills, improve-aeo-crosslink-skill, and improve-404-monitor-skill. PR reuse applied to: improve-drafting-skills, weekly-404-monitor, afdocs-fix, sync-error-docs, sync_terminology, sync-openapi-spec, improve-aeo-crosslink-skill, improve-404-monitor-skill. update-changelog keeps one PR per release (correct) but now detects stacked release PRs. Slack volume: aeo_crosslink_audit no longer posts on no-change runs; weekly-404-monitor gates on threshold and folds its Phase 2 results into a single message instead of two; afdocs-audit posts only on regression or a blocked audit, backed by a new run log for the baseline. Migration artifacts: a find-and-replace during the GitBook-to-Astro move substituted descriptions into file paths. sync-error-docs referenced `astro.config.mjs (sidebar config)` and `vercel.json (redirects)` as real paths, had an invalid grep, and still called the GitBook API - it could not have succeeded. Also corrected the sidebar location to src/sidebar.ts, dropped the GITBOOK_TOKEN dependency, and fixed dead `.warp/references/terminology.md` paths in four skills. Consolidates PRs #450, #454, #468, and #484, which all edited draft_docs/SKILL.md and conflicted with each other. Overlapping patterns were merged rather than stacked, and PR #468's frontmatter-description edits were dropped as already superseded on main. Co-Authored-By: Warp Agent <agent@warp.dev> * docs(skills): document the deployed monthly cron for improve-drafting-skills The schedule was deployed as `0 15 1 * *` (the 1st of each month) rather than the `0 17 * * 1` + first-week-guard combination the skill documented. Both are correct and both fire exactly once a month, but the docs and the deployed schedule disagreed. Documented the deployed expression. Restricting only day-of-month is unambiguous because day-of-week stays `*`, so there is no ORing hazard. The tradeoff is noted: the 1st can land on a weekend, delaying review. Kept the first-week guard as a safety net and explained why, since it no longer trips on its own: it is what would narrow a day-of-week expression back to the first Monday, and it contains the blast radius if the day-of-month/day-of-week ORing mistake is ever reintroduced. Reworded the guard's skip message, which still referenced 'first Monday'. Co-Authored-By: Warp Agent <agent@warp.dev> * docs(skills): treat a log-branch fetch failure as blocked, not a stale fallback Review catch on improve-aeo-crosslink-skill: its step 0 said to fall back to the log copy in the current checkout when the branch fetch fails. That copy comes from `main` — precisely the truncated history the branch read exists to avoid — so the fallback reintroduced the problem this PR set out to fix. The file also contradicted itself: its Slack section already listed 'could not fetch the log branch' as a blocked-run example while step 0 said not to abort. The failure mode is quiet, which is what makes it worth fixing. A short log still parses; only the counts change. The run then either drops below the 8-entry minimum and reports 'too early to analyze', or clears the minimum on stale entries and proposes skill edits from an incomplete picture. Both look like ordinary outcomes, so nobody investigates. Both outer loops that read a log branch now stop before analysis on a fetch failure and post the blocked-run message. improve-drafting-skills already had this behavior documented and is unchanged. Also generalized the rule in the authoring guidelines, since this is a class of bug rather than a one-off: do not adopt a fallback that is quieter but less correct than failing. The test is whether the fallback can change the answer ratherratherratherratherratherratherratherratherratherratherratherratverage transparently — proceeding on one source signal and recording the gap — remain fine, because the reader can see what was missing. Co-Authored-By: Warp Agent <agent@warp.dev> --------- Co-authored-by: Warp Agent <agent@warp.dev>
Patterns addressed
warp-internal/ shipped product behavior.Signal window: last 30 days (2026-07-04 → 2026-08-03).
Primary source: GitHub human review comments and verdicts on agent-coauthored merged PRs (noise from Oz
/oz-reviewtemplate verdicts filtered out).Oz
[SIGNAL:style-lint]/[SIGNAL:pr-review]markers: 0 found in drafting-related run conversations (inner loop still not emitting markers reliably).Standing signal-log PR: #433
Improvement targets
.agents/skills/draft_docs/SKILL.md— require source/live verification for labels/flags/defaults; add durable-UI, availability, team-access, and Related pages rules plus checklist items (applies to all drafting skills that route throughdraft_docs)..agents/templates/feature-doc.md— bracket instructions for platform/preview limits, source-verified procedures, integration admin/access prerequisites, and required Related pages..agents/templates/guide-page.md— requiredVideoEmbedtitle, team-tool access prerequisites, and durable prompt/UI guidance.Patterns reviewed but not acted on
{{TOKEN}}substitution bugs — appears to be a build/transform issue called out in human replies (tokens not substituted), not a missing drafting-skill example; needs eng follow-up rather than another skill bullet.Open questions for human review
Should source verification be a hard blocker (omit unverified claims) even when only a PRD is available andResolved: not a hard blocker. The agent may omit the claim or keep it with an inlinewarp-internalis not checked out?{/* VERIFY: ... */}marker, but reporting is now mandatory —draft_docsstep 9.5 requires every unverified claim to be listed in an## Unverified claimssection of the PR description (or in the response for local sessions), andcreate_prrequires that section even when the list is empty.Test plan
git diff --checkcleanConversation: https://app.warp.dev/conversation/23ca1da7-5292-48dc-bb3d-bb8a337eb730
Run: https://app.warp.dev/conversation/23ca1da7-5292-48dc-bb3d-bb8a337eb730
This PR was generated with Oz.